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PREFACE 

This  report  documents  a briefing  on  the  findings  of  a study  con- 
ducted for  the  Deputy  Chief  of  Staff /Research,  Development,  and  Acquisi- 
tion (AF/RD)  as  part  of  the  project  "Methods  and  Applications  of  Life- 
Cycle  Analysis  for  Air  Force  Systems,"  sponsored  by  Project  AIR  FORCE. 
The  research  was  performed  during  late  1977  and_  the  first  half  of  1978. 
The  F100  Engine  Diagnostic  System  (EDS)  program  characteristics  are 
those  of  October  1978. 

Research  on  the  F100  EDS  program  was  undertaken  to  enhance  the 
understanding  of  how  life-cycle  analysis  is  being  used  in  the  Air  Force 
to  support  system-acquisition  decisions.  Initially,  the  authors  focused 
on  the  analytical  methods  and  procedures  employed  in  the  EDS  life-cycle 
analysis.  When  other  important  substantive  issues  in  the  EDS  program 
became  apparent,  however,  they  expanded  the  inquiry  to  include  experi- 
ence gained  on  other  engine  monitoring  systems  and  its  implications 
for  the  F100  EDS  program. 

This  report  discusses  but  does  not  dwell  on  the  issues  of  life- 
cycle  analysis.  The  primary  emphasis  is  on  the  substance  of  the  EDS 
program,  how  it  relates  to  previous  engine  monitoring  experience,  and 
some  remaining  policy  options. 

The  study  findings  were  presented  in  late  1978  and  early  1979  to 
audiences  in  HQ  USAF,  the  Air  Force  Systems  Command,  the  Air  Force 
logistics  Command,  and  the  Tactical  Air  Command.  The  techniques  de- 
veloped in  the  study  should  also  be  of  interest  to  other  agencies  as 
well  as  to  a wider  audience  of  analysts  and  decisionmakers  concerned 
with  aircraft  turbine  engine  monitoring  or  with  the  broader  topic  of 
life-cycle  analysis. 

A Rand  report  now  in  preparation  describes  in  greater  detail  the 
supporting  research  on  experience  gained  from  selected  engine  monitor- 
ing systems. 


This  briefing  report  examines  the  experience  gained  from  several 
aircraft  turbine  engine  monitoring  systems  used  over  the  last  decade 
and  a half  and  the  implications  of  that  experience  for  a new  monitoring 
system — the  Engine  Diagnostic  System — under  development  for  the  F100 
engine  on  the  F-15  and  F-16  tactical  fighter  aircraft.  The  examination 
reveals  that  two  different  approaches  to  engine  monitoring  have  evolved 
in  attempts  to  achieve  the  goal  of  improved  engine  operations,  mainte- 
nance, and  management  while  reducing  support  costs.  The  first  concen- 
trates on  short-term  operations  and  maintenance  aspects  and  is  usually 
accomplished  by  recording  inflight  data  in  a snapshot  mode,  i.e.,  a 
few  seconds  of  data  either  at  predefined  performance  windows  or  when 
certain  engine  operating  limits  are  exceeded.  The  second  approach 
focuses  on  long-term  design-oriented  benefits  through  improved  knowl- 
edge of  the  engine  operating  environment.  To  achieve  the  design-oriented 
benefits,  data  must  be  recorded  continuously  on  at  least  a few  aircraft 
at  each  operational  location. 

U.S.  monitoring  systems  haVe  focused  on  ^iTjoPt’-t-ernu  maintenance- 
oriented  benefits,  whereas  the  British  have  developed  a system* 
has  focused  only  on  long-term,  design-oriented  benefits.  From  a 
life-cycle  analysis  viewpoint,  it  would  seem  that  both  types  of  bene- 
fits are  worthy  of  consideration  in  any  new  monitoring  system. 

We  have  also  reviewed  ongoing  engine  duty-cycle  research  being 
conducted  by  the  military  services.  This  research  has  demonstrated 
that  neither  the  services  nor  the  engine  manufacturers  have  had  a 
clear  idea  of  engine  operational  usage — i.e.,  of  power  requirements 
and  transients  on  actual  mission  flight  profiles — in  fighter  aircraft. 

As  a result,  engine  parts  life  has  generally  been  overestimated,  and 
expected  life-cycle  costs  have  been  understated.  This  situation  has 
improved  significantly  during  the  past  several  years. 

Much  uncertainty  exists  about  the  benefits  and  costs  attributable 
to  engine  monitoring  systems.  We  believe  that  the  maintenance  cost 
savings  used  to  justify  the  F100  EDS  are  unlikely  to  materialize  over 
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the  short  term.  But  whether  EDS  passes  or  fails  in  the  narrow  sense 
of  cost  savings  over  the  short  term  should  not  be  the  sole  criterion 
on  which  it  is  judged.  The  potential  benefits  of  anticipating  needed 
maintenance,  helping  maintenance  crews  and  engineering  support  person- 
nel to  better  understand  engine  failure  cause  and  effect,  and  verifying 
that  maintenance  has  been  properly  performed  have  substantial  value. 

These  benefits  are  especially  significant  now  that  the  Air  Force  is 
moving  toward  an  on-condition  maintenance  posture,  as  is  envisioned 
for  the  F100  engine.  This  engine  is  also  of  modular  design,  and  so 
the  EDS  can  be  an  important  factor  in  achieving  the  maintenance-oriented 
engine  design  objectives.  Unfortunately,  none  of  these  benefits  can 
be  quantified  on  the  basis  of  experience  to  date. 

The  flight-test  component  of  the  current  development  plan  is  un- 
likely to  yield  conclusive  evidence  on  the  value  of  the  EDS;  too  few 
aircraft  and  flying  hours  are  involved.  Moreover,  the  plan's  narrow 
focus  on  hardware  omits  expedient  consideration  of  EDS  as  an  operating 
system,  including  software  and  data-processing  needs.  Before  the  EDS 
is  committed  to  flight  test,  the  scope  of  the  program  should  be  broad- 
ened to  include  consideration  of  the  valuable  long-term  contribution 
that  continuous  recorded  data  can  make  to  engine  designers.  It  is 
particularly  important  to  provide  a means  for  determining  the  correlation 
between  testing  and  operational  duty  cycles  for  the  engines  in  the  F-15 
and  F-16  aircraft.  The  mission  profiles  of  these  two  aircraft  will  be 
quite  different  and  their  engines,  although  models  of  the  F10Q,  will 
be  subjected  to  different  use  and  should  therefore  be  tested  to  the 
relevant  duty  cycles.  Such  information  should  help  the  Air  Force  in 
its  component  improvement  program  for  the  F100  engine,  as  well  as  in 
future  engine  design  programs,  especially  now  that  reliability,  dura- 
bility, and  cost  issues  are  almost  on  an  equal  footing  with  performance 
objectives. 

We  believe  that  the  Air  Force  should  continue  to  develop  a turbine 
engine  monitoring  system  for  the  F100  engine  but  recommend  that  it  take 
certain  actions  before  committing  a quarter  of  a billion  dollars 
(1977  $)  to  outfit  all  F-15  and  F-16  aircraft  with  EDS.  First,  we 
recommend  that  a flexible  phased-implementation  schedule  be  devised 
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that  will  expand  the  test  program  and  defer  a full-scale  production 
go-ahead.  Second,  for  both  the  F-15  and  F-16  applications,  we  recom- 
mend that  a continuous  recording  option  be  developed  to  provide  the 
necessary  design  data  and  that  a small  staff  be  dedicated  to  analyzing 
these  data.  Third,  we  recommend  that  the  life-cycle  analysis  be  re- 
vised to  provide  a more  realistic  estimate  of  the  costs  and  benefits 
for  various  EDS  options. 
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I.  INTRODUCTION 

During  the  past  several  years,  Rand  has  been  studying  methods  and 
applications  of  life-cycle  analysis  for  the  United  States  Air  Force. 
Life-cycle  analysis  of  weapons  systems  programs  requires  consideration 
of  the  benefits , which  cannot  always  be  fully  quantified  (and  in  the 
military  certainly  not  in  the  same  metric  as  cost) , as  well  as  the 
costs,  which  appear  in  most  cases  to  be  more  readily  quantifiable. 

Our  overall  research  objectives  are  (a)  to  improve  the  analytical 
methods  applicable  to  life-cycle  analysis  and  (b)  to  assess  and  improve 
the  scope  and  richness  of  information  provided  to  high-level  decision- 
makers in  support  of  life-cycle  analysis  studies  and  recommendations. 
Several  recent  Air  Force  programs  in  which  life-cycle  analysis  was  a 
major  feature  were  selected  as  examples  for  our  research.  One  of  these 
exemplary  programs,  the  F100  Engine  Diagnostic  System  (EDS),  is  the 
topic  of  this  briefing  report. 

In  presenting  the  briefing,  we  tried  to  combine  features  of  the 
broader  methodological  objectives  of  life-cycle  analysis  with  specific 
applications  relevant  to  the  ongoing  F100  EDS  program.  Thus,  our 
presentation  had  two  objectives:  (1)  to  discuss  some  life-cycle  anal- 
ysis issues,  and  (2)  to  present  findings  that  apply  directly  to  the  F100 
EDS  program.  One  issue,  for  example,  concerns  the  potential  long-term 
benefits  of  the  EDS  that  are  applicable  to  the  F100  but  have  not  been 
considered  previously. 

APPROACH  TO  THE  STUDY 

The  analytical  approach  used  in  this  study  was  to  examine  the  ex- 
perience of  several  selected  engine  monitoring  systems,  and  then  use 

* 

that  information  as  a basis  for  evaluating  the  F100  EDS.  The  scope 
of  the  briefing  is  outlined  in  Fig.  1.  We  will  review  the  F100  EDS  to 
provide  a background'  and  context  for  the  findings  obtained  from  our 


A Rand  report  now  in  preparation  describes  in  greater  detail  the 
supporting  research  on  experience  gained  from  selected  engine  monitor- 
ing systems. 
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Figure  1 

examinations  and  analyses  of  the  selected  engine  monitoring  systems. 
The  systems  examined  included:  (1)  Time  Temperature  Recorder  (TTR) , 

(2)  Engine  Health  Monitoring  System  (EH1IS) , (3)  Malfunction  Detection 
Analysis  Recording  System  (MADAKS),  (4)  Inflight  Engine  Condition 
Monitoring  System  (IECMS) , (5)  Airborne  Integrated  Data  System  (AIDS), 
and  (6)  Engine  Usage  Monitoring  System  (EUMS).*  The  EUMS  is  a British 
system  that  has  a different  focus  from  that  of  the  U.S.  systems.  Re- 
cent research  on  engine  duty-cycle  analysis  was  also  examined.^  Engine 
duty-cycle  analysis  is  of  particular  interest  because  it  has  a signifi- 
cant effect  on  engine  design  and  testing.  The  findings  derived  from 
both  the  monitoring  system  experience  and  the  duty-cycle  analysis  will 
be  discussed  in  terms  of- their  implications  for  the  F100  EDS. 

* 

16  The  details  of  these  case  studies  are  given  in  Refs.  1 through 
+See  Refs.  13,  17,  and  18. 
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Previous  F100  EDS  studies  have  focused  mainly  on  net  life-cycle 

cost  savings,  obtained  through  estimated  reductions  in  maintenance 

labor,  spare  parts,  and  fuel  usage,  as  justification  for  developing  the 
* 

system.  The  intent  here  is  not  to  challenge  the  cost-estimating  pro- 
cedure in  general,  or  any  particular  cost  value  used  in  the  life-cycle 
cost  calculations,  but  rather  to  focus  on  the  comprehensiveness  of  the 
cost  elements  considered,  the  expectation  of  achieving  the  net  cost 
savings  estimated,  and  the  type  of  information  presented  to  high-level 
decisionmakers  leading  to  the  EDS  decisions.  Along  with  the  cost  esti- 
mate, the  F100  EDS  development  plan  and  its  design  objectives  were  also 
examined . 

In  these  three  areas  (design  objectives,  development  plan,  costs), 
there  are  substantial  problems  because  the  original  formulation  of  the 
F100  EDS  concept  did  not  take  advantage  of  lessons  learned  from  previ- 
ous monitoring  experience.  This,  in  turn,  led  to  an  incomplete  and 
perhaps  potentially  misleading  estimate  of  life-cycle  costs. 

The  body  of  this  report  provides  the  rationale  for  our  conclusions 
and  recommendations.  Substantive  changes  to  the  program  are  recommended. 
Fortunately,  the  program  is  in  its  infancy,  and  important  decision 
points  affecting  its  development  and  procurement  still  lie  ahead. 

THE  F100  EDS  PROGRAM 

The  F100  engine  is  the  most  advanced  and  most  expensive  fighter 
engine  in  use  today.  It  is  of  modular  design,  and  the  USAF  intends  to 
employ  the  "on-condition  maintenance"  concept  in  managing  the  engine, 
i.e.,  to  perform  maintenance  based  on  positive  identification  of  con- 
dition rather  than  an  operating  time  limit. ^ This  represents  a marked 
change  from  the  present  "hard  time"  maintenance  approach,  i.e.,  mainte- 
nance performed  at  fixed  operating  time  intervals.  For  "on-condition 
maintenance"  to  work  effectively,  the  condition  of  the  engine  must  be 
known  with  a much  improved  degree  of  precision,  and  the  relationship 
between  engine  parameter  trends  and  failure  modes  must  be  established. 

The  F100  EDS  program  (Fig.  2)  grew  out  of  this  need  to  develop  new 

* 

See  Ref.  19. 

^For  a more  complete  discussion,  see  Ref.  20,  pp.  57-59. 
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DESCRIPTION  OF  ROO  EDS  PROGRAM 


DESIGN  OBJECTIVES:  • INCREASE  AIRCRAFT  SYSTEM 
AVA1LABIUTY/REU  ABILITY 

• REDUCE  LIFE  CYCLE  COST 

• IMPROVE  MANAGEMENT  OF  F1CO  ENGINE  PROGRAM 

DEVaOPMENT  PLAN:  • CURRENTLY  IN  ENGINEERING  DEVaOPMENT  LEADING 
TO  FLIGHT  TEST  DEMONSTRATION 

- 5 F-15S  (10  MONTHS) 

- STARTS  JAN  '79 

- COMPLETION  NOV '79 

• PRODUCTION  DECISION  EXPECTED  EARLY  IN  1980 

SCOPE:  • EQUIP  ALL  F-)5s  & F-16s  (2000  AIRCRAFT 73000  ENGINES) 

• ROT&E  AND  PROCUREMENT  COST 
*280  MILLION  ($19771 


Figure  2 


propulsion  operations  and  maintenance  procedures  and  management  prac- 

* 

tices  while  reducing  support  costs.  Because  of  this  need,  the  Air 
Force  established  design  objectives  for  the  EDS  that  are  intended  to 
benefit  the  FlOO  engine  by  collecting  inflight  engine  data. 

The  EDS  derives  data  from  about  40  engine  senscrs  designed  to 
measure  pressures,  temperatures,  speeds,  control  positions,  and  vibra- 
tion. Additional  airframe  sensors  provide  data  on  altitude,  angle  of 
attack,  airspeed,  fuel  flow,  and  other  relevant  parameters.  The  data 
are  recorded  in  two  modes.  In  the  first  mode,  the  EDS  employs  a snap- 
shot recording  of  inflight  data.  Engine  and  airframe  data  are  con- 
tinuously monitored  and  stored  for  the  preceding  5 seconds.  An  out- 
of-limit  event  triggers  data  recording  for  the  5-second  period 
immediately  preceding  the  event  and  for  2 seconds  afterward.  The 
system  also  records  snapshot  data  at  predetermined  mission  windows 


Reference  21  provides  insight  into  the  potential  maintenance 
benefits  as  perceived  by  the  user,  the  Tactical  Air  Command. 

% 
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(altitude/Mach  number  sustained  for  a specific  period  of  time)  for 
postflight  analysis  and  the  trending  of  data.  These  data  are  intended 
to  permit  the  trending  of  engine  characteristics  and  the  diagnosis  of 
engine  problems.  In  the  second  mode,  the  EDS  cumulatively  counts  and 
records  Major  engine  cycles  and  time  above  specified  critical  tempera- 
tures. More  accurate  estimates  of  the  remaining  useful  life  of  engine 
parts  and  better  inventory  and  maintenance  management  are  anticipated. 

The  F100  EDS  program,  currently  in  engineering  development,  will 
lead  to  a demonstration  program  on  five  F-15s  flown  for  a 10-month 
period.  The  flight-test  demonstration  is  expected  to  be  completed  in 

November  1979,  and  to  be  followed  by  a production  decision  in  early 

* 

1980.  This  plan  is  central  to  certain  important  points  that  we  will 
make  regarding  lessons  learned  from  previous  experience. 

The  scope  of  the  F100  EDS  program  is  large.  It  involves  all  the 
F-15  and  F-16  tactical  fighter  aircraft,  and  affects  approximately 
2000  airframes  and  3000  FI 00  engines.  The  RDT&E  and  procurement  costs 
for  the  EDS  are  estimated  to  be  about  $280  million  (FY1977  dollars) . 

The  current  Air  Force  life-cycle  analysis  suggests  that  if  the  stated 
design  objectives  are  met,  the  F100  EDS  investment  costs  will  be  fully 
recovered  through  net  savings  of  outyear  engine  recurring  support  costs 
during  the  useful  life  of  the  engine. 


As  of  October  1978;  more  recent  information  indicates  that  the 
schedule  is  expected  to  change. 
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II.  SELECTED  CASE  STUDIES 


The  selected  case  studies,  shown  in  Fig.  3,  reflect  Air  Force, 

Navy,  commercial,  and  British  applications  of  engine  monitoring  systems. 
The  systems  were  applied  to  engines  for  U.S.  military  fighter,  attack, 
trainer,  and  cargo  aircraft,  commercial  transports,  and  a spectrum  of 
British  aircraft.  Applications  include  both  single-pilot  and  multi- 
crew aircraft  and  single-engine  and  multiengine  designs. 


CASE  STUDIES 

SYSTEM 

A IRCRAFTI  ENGINE 

COMMENT 

TIME  TEMPERATURE  RECOROER  (TTR) 

SEPT  67  - JAN  69 

F-1007J57 

SINGLE  PARAMETER 

MEASUREMENT;  CONUS  & 

VIET  NAM  EXPERIENCE 

IC0NTR0L  GROUP) 

ENGINE  HEALTH  MONITORING  SYSTEM 

(EHMSIjULY  76  . my  v 

T-38/J85 

FLEET  RETROFIT  NOT 

RECOMMENDED 
(CONTROL  GROUP) 

MALFUNCTION  DETECTION  ANALYSIS 
RECORDING  SYSTEM  (MADARS) 

LATE  60' s - PRESENT 

C-5A/TF39 

CONTINUOUS  RECORDINGS 

USED  TO  EXTEND  TF39  TBO 

AND  FOR  CONFIGURATION 
MANAGEMENT 

IN-FLIGHT  ENGINE  CONDITION 
MONITORING  SYSTEM  (IECMS) 

1973  - PRESENT 

A-7E/TF41 

USEFUL  FOR  OPERATIONS  & 
COMPONENT  IMPROVEMENT 

PROGRAM:  STILL  IN  DEVELOP- 
MENT [NO  CONTROL  GROUP) 

AIRBORNE  INTERGRATED  DATA  SYSTEM 
(AIDS) 

IA1E  60's  - PRESENT 

COMMERCIAL 

WIDEBODIES 

LIMITED  APPLICATIONS; 

RETROFIT  NOT  COST 

EFFECTIVE 

ENGINE  USAGE  MONITORING  SYSTEM 
(EUMS) 

EARLY  70's  - PRESENT 

U.K.  AIRCRAFT 

INTENDED  TO  REDUCE 

FUTURE  LCC  THROUGH 

IMPROVING  ENGINE  DESIGN 

Figure  3 


The  monitoring  systems  themselves  ran  the  gamut  of  parameter 
measurement — the  TTR  system  measures  only  a single  parameter,  whereas 
IECMS  measures  in  excess  of  60  parameters.  Most  of  the  systems  used 
snapshot  recording,  very  much  like  the  F100  EDS;  others  recorded  data 
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continuously . The  operational  focus  of  the  monitoring  systems  was  also 
varied.  The  U.S.  systems  are  oriented  primarily  toward  improving  day- 
to-day  maintenance,  whereas  the  British  system  ignores  the  short-term 
maintenance  benefits,  choosing  rather  to  emphasize  the  longer-term 
feedback  of  operational  data  to  the  design  and  test  communities.  An 
identified  control  group  existed  for  several  of  the  monitoring  systems. 
Unfortunately,  the  control  groups  did  not  control  for  all  the  variables 
of  interest,  and  the  time  interval  for  most  tests  was  too  short  to  sta- 
bilize inputs  and  quantify  some  of  the  possible  outcomes.  Nevertheless 
although  all  the  information  desired  isn't  available,  much  useful  inf or 
mation  was  obtained. 

The  study  described  here  drew  upon  the  experience  from  all  the 
case  studies,  but  for  this  summary  report  we  have  selected  two  case 
studies  as  illustrative  examples:  the  EHMS  on  the  T-38  because  it  is 
the  U.S.  AF  system  that  most  closely  resembles  the  approach  taken  by 
the  F100  EDS,  and  the  British  EUMS  because  the  approach  taken  is  en- 
tirely different  from  that  of  the  U.S.  systems.  These  studies  will 
be  discussed  below. 

THE  T-38  EHMS  CASE  STUDY 

The  T-38  EHMS  is  the  Air  Force  system  most  similar  to  the  F100 
EDS.  It  emphasizes  improved  day-to-day  engine  maintenance.  Engine 
health  data  are  stored  only  under  the  following  three  conditions:  (1) 
when  engine  parameters  exceed  normal  limits,  (2)  on  pilot  command,  and 
(3)  under  preprogrammed  flight  conditions.  When  any  of  the  three  con- 
ditions occur,  all  parameter  data  as  of  that  moment  are  recorded  (snap- 
shot recording) . Program  data  were  obtained  for  the  test  that  was 

* 

conducted  from  July  1976  to  May  1977.  Two  groups  of  engines  were  used 
in  the  test:  an  instrumented  group  and  a control  group  (see  Fig.  4). 
The  two  groups  contained  the  same  number  of  engines  and  were  used  for 
approximately  the  same  number  of  flight  hours.  Statistical  testing 
detected  no  significant  difference  in  time  since  last  overhaul  or  in 
flight-hour  distributions  among  the  engines  of  both  groups. 


The  data  for  the  instrumented  and  control  engines  are  summarized 
by  engine  serial  number  in  Ref.  11.  Individual  engine  malfunction  re- 
port data  were  obtained  from  the  Program  Office. 
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EHMS  (T-38/J85)  ACTIVITY  OUTCOMES 

JULY  76  - MAY  77 


INSTRUMENTED  ENGINES 

CONTROL  ENGINES 

NUMBER  OF  ENGINES 

26 

26 

TOTAL  FLIGHT  HOURS 

6226 

6443 

MALFUNCTION  REPORTS 

97 

48 

GROUND  MAINTENANCE 

UNSCHEOULID  REMOVALS 

53 

23 

TROUBLESHOOTING  (MH) 

169 

90 

REPAIR  IMH) 

1403 

530 

ENGINE  GROUND  RUNS 

TROUBLESHOOTING 

52 

38 

TRIMS 

26 

14 

FUEL  USED  (GAL) 

TROUBLESHOOTING 

4846 

1786 

TRIMS 

5720 

4480 

Figure  4 


Twice  the  number  of  malfunctions  were  reported  for  the  instru- 
mented engines  as  for  the  control  engines.  This  resulted  in  a larger 
number  of  malfunction  reports,  which,  in  turn,  increased  the  number 
of  unscheduled  engine  removals,  troubleshooting  and  repair  manhours, 
the  number  of  ground  runs,  and  the  amount  of  fuel  used  for  trouble- 
shooting and  trimming  the  engines.  Examination  of  the  malfunction 
reports  revealed,  however,  that  the  EHMS  flagged  only  five  malfunctions 

out  of  a total  of  97,  independent  of  pilot  and  maintenance  crew  reports. 

+ 

From  discussions  with  those  involved  in  the  test,  and  an  analysis  of 
the  data,  we  conclude  that  the  increased  number  of  malfunction  reports 
resulted  from  increased  sensitivity  on  the  part  of  the  pilot  and  main- 
tenance crew.  Both  pilot  and  maintenance  crew  knew  that  the  system 


More  recent  data  from  F-5  experience  at  Holloman  AFB,  where  the 
mission  profile  is  more  severe,  indicate  that  the  system  finds  more 
engine  problems  independently. 

+ 

See  Ref.  1. 
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was  on  the  aircraft,  and  their  behavior  was  affected  by  that  knowledge. 
Although  the  EHMS  reported  few  malfunctions  independently,  it  did 
provide  information  that  significantly  improved  the  cause-and-eff ect 
understanding  of  engine  problems  encountered  and  that  was  useful  in 
providing  maintenance  direction  to  the  ground  crews. 

It  is  interesting  to  note  that  the  J85  was  a mature  engine  and 
that  the  number  of  engine  problems  encountered  was  not  very  great. 

Also,  the  EHMS  was  not  a new  system,  having  been  through  a feasibility 
phase  during  December  1972  and  January  1973.  Nevertheless,  hardware 
and  software  problems  in  the  EHMS  caused  schedule  delays.  A 6-month 
delay  was  necessary  to  shake  down  the  installation  and  software.  The 
shakedown  not  only  lowered  the  false-alarm  rate  but  also  reduced  the 
EHMS  maintenance  manhours  per  flight  hour  below  the  number  deemed 
necessary  for  day-to-day  operation. 

The  EHMS  Program  Office  had  anticipated  that  less  maintenance, 
less  fuel,  and  fewer  problems  would  result  in  all  the  categories  shown 
in  Fig.  4,  but  more  manhours  and  fuel  were  actually  required,  result- 
ing in  higher  cost.  Confusion  exists  as  to  the  costs  and  benefits 
that  might  reasonably  be  expected  from  an  engine  diagnostic  system. 

All  the  potentially  important  system  and  engine  benefits — improved 
readiness,  availability,  reliability,  and  lower  intermediate  and  depot 
maintenance  costs — were  not  demonstrated  because  the  controls  necessary 
to  the  collection  of  data  appropriate  for  analysis  were  not  maintained 
over  a sufficiently  long  test  period.  Current  objectives  of  monitor- 
ing systems  are  oriented  to  cost-reduction,  and  the  lack  of  control 
for  certain  potentially  important  variables  results  from  a lack  of 
emphasis  on  noncost  benefits.  In  spite  of  the  additional  unscheduled 
engine  removals,  maintenance  manhours,  and  fuel  used,  no  measurable 
positive  output — i.e.,  no  cost  savings  and  no  increase  in  readiness 
or  availability — was  discernible  from  the  EHMS  experience.  Perhaps 
the  engine's  maturity  and  the  shortness  of  the  test  explain  the  lack 
of  positive  outputs. 

THE  BRITISH  EUMS  CASE  STUDY 


The  British  system  (see  Fig.  5)  purposely  ignores  the  short-term 
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U.K.  APPROACH  - ENGINE  USAGE  MONITORING  SYSTEM  (EUMS) 


OBJECTIVES:  - IMPROVE  UNDERSTANDING  OF 

ENGINE  DUTY  CYCLE /MISSION 

APPROACH:  - INSTRUMENT  SMALL  SAMPLE  OF 

EACH  TYPE  AIRCRAFT 

- RECORD  PARAMETERS  CONTINUOUSLY 

- UTILIZE  DEDICATED  ENGINEERING  STAFF 
-ANALYZE  DATA  BY  MISSION  TYPE 

EXPECTED  BENEFITS:  -ENGINE  DESIGN 

-CORRELATE  TEST /DUTY  CYCLES 

-MAINTENANCE 

-RELIABILITY 

-COMPONENT  IMPROVEMENT  PROGRAM  (CIP) 


Figure  5 


maintenance-oriented  benefits;  instead,  the  British  choose  to  concen- 
trate on  perfecting  their  engineering  understanding  of  the  engine 
operating  environment  and  to  uge  this  understanding  to  improve  present 
and  future  engine  designs.  The  system's  primary  objective  is  to  im- 
prove safety  and  availability  and  ultimately  to  reduce  life-cycle  cost. 

The  focus  of  the  EUMS  reflects  two  conclusions  that  the  British 
reached  in  the  late  1960s  and  early  1970s  when  they  experienced  a 
spate  of  engine  problems  coupled  with  rising  operating  and  support 
costs.  Preliminary  investigations  revealed  that  the  problems  occurred 
because  the  engine's  operational  environment  was  not  completely  under- 

Ac 

stood.  Subsequent  studies  concluded  that  a system  that  would  provide 
a continuous  recording  of  a few  parameters  was  the  simplest,  cheapest, 
most  reliable,  and  most  productive  solution  available,  given  the 

* 

The  EUMS  is  also  partially  a result  of  the  difficulty  that  the 
British  had  in  understanding  how  the  Rolls  Royce  Spey  engine  could  be 
so  successful  in  the  F-4K  interceptor,  and  in  a modified  version  (the 
TF41)  this  engine  could  be  so  troublesome  in  the  A-7E  attack  aircraft. 
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prevailing  state  of  the  art.  This  conclusion  was  also  influenced  by 

the  fact  that  despite  the  investments  made  by  the  U.S.  in  complex 

monitoring  programs,  there  was  not  a single  one  that,  the  British  felt, 

* 

had  explored  the  full  potential  of  engine  health  monitoring. 

The  British  proceeded  to  develop  a continuous  recorder  that  would 
record  a small  number  of  engine  parameters.  For  example,  on  the  Harrier 
engine,  only  eight  parameters  are  recorded.  This  equipment  is  being 
installed  on  a few  of  each  type  of  aircraft  now  in  operation.  The 
field  data  are  sent  directly  to  a central  location,  where  they  are 
analyzed  by  a dedicated  engineering  staff  consisting  of  engineers  from 
the  Ministry  of  Defence  and  from  the  testing  community,  as  well  as  by 
engineers  from  Rolls  Royce.^  Analyzed  by  mission  type,  these  data 
have  yielded  interesting  and  important  observations.  The  British  have 
found,  for  example,  that  the  amount  of  engine  life  consumed  depends, 
to  an  important  extent,  on  the  type  of  missions  flown  and  how  the 
equipment  is  used  during  the  mission.  They  also  found  that  a major 
contributor  to  reduced  engine  life  is  the  cumulative  effect  of  small 
power  transients.  They  have  concluded — and  the  U.S.  military  services 
are  also  reaching  the  same  conclusion — that  engine  failure  modes,  such 
as  low-cycle  fatigue,  have  not  been  as  well  understood  as  they  were 
thought  to  be.  Quantitative  engineering  data  are  now  being  used  to 
improve  engine  design  specifications  and  to  bring  both  full-scale  and 
component  test  cycles  in  line  with  operational  duty  cycles.  The 
British  are  in  the  process  of  reorienting  their  approach  to  maintain- 
ability and  reliability,  recognizing  that  these  areas  are  more  a func- 
tion of  engine  throttle  cycles  experienced  and  the  type  of  mission 
flown  than  of  flying  hours  only. 

ENGINE  MONITORING  SYSTEM  OUTCOMES 

To  evaluate  the  strengths  and  weaknesses  of  previous  monitoring 
systems,  we  divided  system  objectives  into  two  groups  based  on  a time 

?Hf 

See  Refs.  6,  7,  and  9 for  a complete  discussion  of  the  EUMS  ob- 
jectives and  approach. 

+ 

The  British  method  of  handling  data  is  designed  for  an  engineer- 
ing study.  For  day-to-day  maintenance  purposes  data  must  be  used  at 
the  squadron  level  in  the  field.  This  is  an  important  distinction. 
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orientation:  (1)  short-term  operations,  maintenance,  and  management 

and  (2)  long-term  design.  Figure  6 lists  the  characteristics  that  we 
feel  are  desirable  in  a monitoring  system.  Certain  of  these  charac- 
teristics served  as  design  objectives  for  each  of  the  case  studies. 
For  example,  as  a result  of  engine  system  monitoring,  EHMS  expected 

fewer  maintenance  manhours,  a savings  in  fuel,  fewer  parts  consumed, 

* 

and  fewer  unscheduled  engine  removals. 


SUMMARY  OF  ENGINE  MONITORING  SYSTEMS  OUTCOMES 


MAINTENANCE  ORIENTED 

• OPERATIONAL 

- AWARE  OF  ENGINE  HEALTH 

- AWARE  OF  ENGINE  OVERTEMPERATURES 

• MAINTENANCE 

V-  LESS  MAINTENANCE  MANPOWER 
V - LESS  TROUBLESHOOTING  & TRIM  FUEL 
v - LESS  ENGINE  REMOVALS 
V-  LESS  PARTS  CONSUMPTION 
-ANTICIPATE  MAINTENANCE  (TRENOINGI 

- IMPROVE  CAUSE  & EFFECT  UNDERSTANDING 

- VALIDATE  MAINTENANCE  ACTION 

• MANAGEMENT 
V- MODIFY  TBO 

- PROVIDE  CONFIGURATION  CONTROL 
DESIGN  ORIENTED 

- GUIDE  CIP 

- CORRELATE  TEST  / DUTY  CYCLES 

- AID  FUTURE  ENGINE  DESIGN 


Figure  6 


Since  the  primary  design  objective  of  most  monitoring  systems  is 
to  reduce  resources,  the  five  outcomes  noted  by  checkmarks  in  Fig.  6 
have  received  the  most  attention  because  it  is  easier  to  cost  them. 

«Jp 

The  background  of  EHMS  and  details  of  the  benefits  expected  from 
engine  monitoring  are  discussed  in  Ref.  11. 
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Given  sufficient  information,  all  characteristics  could  be  assigned 
values.  The  first  four  characteristics  were  important  in  providing 
justification  for  the  F100  EDS,  but  ail  of  them  must  be  considered  be- 
cause, in  some  cases,  the  unchecked  characteristics  may  justify  the 
costs  of  a new  monitoring  system. 

To  show  the  outcome  of  each  monitoring  system,  we  employ  the 
matrix  shown  in  Fig.  7.  (The  EDS  column  will  be  filled  out  in  Fig. 
10.)  The  coding  used  in  Fig.  7 requires  some  explanation. 


SUMMARY  OF  ENGINE  MONITORING  SYSTEMS  OUTCOMES 


MAINTENANCE  ORItNTED 
• OPERATIONAL 


' AWARE  OF  ENGINE  HEALTH 
' aware  OF  ENGINE  0VERTEMPERA1URES 


• MAINIENANCE 


✓ 

1 


-LESS  MAINTENANCE  MANPOWER 
-LESS  TROUBLESHOOTING  *.  TRIM  FUEL 

— FEWER  ENGINE  REMOVALS 
-LESS  PARTS  CONSUMPt ION 
-ANTICIPATE  MAINTENANCE  (TRENOINGl 

- IMPROVE  CAUSE  & EFFECT  UNDERSTANDING 
-VALIDATE  MAINTENANCE  ACTION 


• MANAGEMENT 


w'  - MODIFY  TBO 

- PROVIDE  CONFIGURATION  CONIKUl 
DESIGN  ORIENTED 


- GUIDE  CIP 

- CORRELATE  TEST  /DUTY  CYCLES 
-AID  FUTURE  ENGINE  DESIGN 


SBTI 


Figure  7 


The  red  coding  means  that  the  particular  characteristic  was  not 
achieved;  but  we  must  point  out,  however,  that  in  many  cases  it  was 
not  a design  objective  and  no  attempt  was  made  to  achieve  it.  The 
yellow  coding  signifies  that  the  information  necessary  to  determine 
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lf  the  monitoring  system  did  or  did  not  accomplish  the  characteristic 
is  lacking.  The  green  coding  indicates  that  the  characteristic  was 
achieved  or  was  beginning  to  be  achieved.  Several  of  the  green  blocks 
are  crosshatched  to  indicate  that  the  benefits  existed  but  additional 
explanation  is  required. 

Moving  down  the  EHMS  column,  we  see  that  both  operational  charac- 
teristics were  achieved.  Becaused  over temperature  damages  the  most 
critical  and  most  expensive  engine  components,  this  information  is 
extremely  important.  Our  research  on  engine  overtemperatures  in  fighter 
and  attack  aircraft  shows  that  pilots  report  only  about  one-third  of 
the  significant  engine  overtemperatures  and  are  unable  to  report  over- 
temperature duration.  This  is  to  be  expected,  because  these  aircraft 
are  mostly  single-crew  aircraft,  and  the  pilot  sits  in  a small,  crowded, 
vibrating  cockpit  with  a number  of  small  gauges.  His  attention  must 
be  divided  among  many  functions,  whereas  the  monitoring  system  watches 
engine  parameters  full  time. 

For  the  EHMS,  the  first  three  characteristics  under  maintenance 
are  coded  red  because  the  instrumented  engines  consumed  more  maintenance 
manhours,  troubleshooting,  and  trim  fuel,  and  experienced  a higher  re- 
moval frequency,  than  the  control  group  engines  during  the  flight-test 
evaluation.  The  parts-consumption  block  is  colored  yellow  because  no 
data  were  collected. 

The  last  three  characteristics  under  maintenance  are  colored  green 
because  they  were  just  beginning  to  be  achieved,  but  achievement  re- 
quires time  to  develop  fully.  Looking  at  the  lower  left-hand  corner 
of  the  matrix,  we  see  that  the  management  and  the  long-term  design- 
oriented  blocks  for  both  the  TTR  recorder  and  the  EHMS  are  red.  Al- 
though the  EHMS  was  not  intended  to  achieve  the  long-term  benefits,  we 
think  that  the  matrix  is  also  telling  us  that  when  a diagnostic  system 
is  applied  to  a mature  engine,  it  does  not  have  an  opportunity  to  make 
a substantial  impact  in  these  two  areas.  This  is  because  the  engine 
is  well  understood  and  most  of  the  problems  have  been  corrected,  or  its 
service  life  is  considered  satisfactory. 

In  contrast,  MADARS,  IECMS,  and  AIDS  were  all  installed  either  as 
original  equipment  or  very  early  in  engine  life.  Each  system  records 
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internal  gas  temperatures  and  pressures.  Since  all  of  them  provide  the 
operator  with  an  awareness  of  engine  health,  these  blocks  are  coded 
green.  Under  maintenance,  however,  the  first  four  characteristics  are 
ceded  yellow  because  either  all  of  these  engines  are  instrumented  or 
there  never  has  been  an  identified  control  group. 

The  improved  awareness  and  understanding  provided  by  the  monitor- 
ing systems  has  an  important  effect  on  the  last  three  maintenance  objec 
tives.  Three  examples  are:  (1)  In  the  case  of  hard  time  maintenance 
procedures,  the  data  can  be  used  to  study  engine  parameter  trends. 

Some  maintenance  problems  can  then  be  alleviated  by  using  a different 
maintenance  schedule.  (2)  The  data  will  improve  the  maintenance  crew’s 
understanding  of  engine  malfunction  cause  and  effect,  permitting  the 
crew  to  work  smarter  rather  than  harder.  (3)  The  data  can  be  used  to 
avoid  a shotgun  approach  to  maintenance  in  which  good  parts  are  need- 
lessly replaced. 

Unfortunately,  it  takes  time  to  fully  achieve  these  benefits. 
Experiencing  engine  failures,  and  reviewing  the  data  leading  up  to 
the  failures,  precedes  any  attempt  to  correlate  a particular  engine 
parameter's  trend  with  an  incipient  engine  failure.  Only  when  such 
correlation  exists  can  the  operations,  maintenance,  and  management 
personnel  make  use  of  this  information  to  schedule  or  initiate  mainte- 
nance. 

In  the  case  of  modifying  the  Time  Between  Overhaul  (TBO)  for  the 
MADARS,  the  crosshatched  green  block  indicates  that  the  monitoring 
system  data  provide  one  of  many  inputs  that  constitute  the  information 
set  enabling  TBO  to  be  extended.  The  technical  data  provided  by  the 
monitoring  system  helps  to  establish  the  proper  TBO  in  three  ways: 

(1)  by  providing  additional  confidence  to  the  decisionmaking  process; 

(2)  by  helping  to  uncover  incipient  failure  modes  that  are  at  present 
undetected  in  C-5A  fleet  engines;  (3)  by  tracking  the  reliability  and 
durability  of  new  parts  and  components  incorporated  into  the  engines. 

For  the  IECMS,  the  green  crosshatching  shows  that  the  long-term 
design-oriented  benefits  were  not  part  of  the  original  monitoring  ob- 
jectives but  resulted  from  an  engineering  need  to  better  understand 
engine  operational  use.  The  IECMS  program  developed  a continuous 
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recording  option  to  enhance  this  understanding.  The  continuously 
recorded  data  permitted  the  operational  duty  cycle  to  be  correlated 
with  appropriate  testing.  This  is  an  extremely  important  contribution, 
because  unless  correlation  exists , operational  fleet  failure  modes 
may  not  be  reproduced  in  the  test  cell . 

The  matrix,  then,  reveals  that  the  U.S.  monitoring  systems  have 
focused  on  the  short-term  maintenance-oriented  benefits.  In  contrast 
to  the  U.S.  systems,  the  British  EUMS  has  ignored  the  short-term 
maintenance-oriented  benefits,  choosing  instead  to  focus  on  the  longer- 
term  design,  testing,  and  management  benefits.  The  British  system  is 
different  in  a way  that  we  think  is  important:  it  focuses  on  improv- 
ing engine  design, 

ENGINE  DUTY-CYCLE  RESEARCH 

The  U.S.  Navy  and  Air  Force,  independent  of  the  British,  have 
also  experienced  many  severe  engine  problems  and,  from  p eliminary 
investigations,  have  concluded  that,  their  engineering  understanding 
of  the  engine  operational  environment  has  been  inadequate.  For  ex- 
ample, the  type  of  early  information  supplied  to  the  engine  manufac- 
turer for  design  guidance  for  a new  engine  is  shown  in  Fig.  8 for  the 
* 

Navy  F-14,  The  estimated  power  required  as  a function  of  time  for 
the  F-14  intercept  mission  is  shown  in  view  (a) . The  actual  number 
of  engine  cycles  that  occurred  on  an  instrumented  F-14  during  flight 
is  shown  in  view  (b) , together  with  the  resulting  change  in  the  pre- 
dieted  design  life  of  certain  important  engine  components.  Low-cycle 
fatigue  is  important  because  cumulative  fatigue  damage  occurs  in  cy- 
clicly  loaded  parts  as  these  parts  are  cycled  from  low  to  maximum  RPM. 
Current  methods  for  calculating  LCF  rely  on  the  usage  rates  of  cycles 
per  hour  that  are  derived  from  synthetic  sortie  patterns, 

*Propulsion  systems  are  designed  to  Request  for  Proposal  (RFP) 
mission  profDes  based  on  a projected  combat  time  with  specific  weap- 
ons load  and  avionics  suite. 

+ 

The  impact  of  engine  duty  cycle  on  part  life  and  maintenance 
intervals  is  discussed  in  Refs,  17  and  18. 

*The  sortie  pattern  is  not  the  only  variable.  For  instance,  one 
example  of  recent  IECMS  data  shows  a significant  difference  in  engine 
usage  by  a flight  leader  and  wing  man  during  the  same  mission. 
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(a)  Estimated  power  (EFFj  required  ai  a function  of 
mission  time  for  an  intercept  mission. 
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(b)  Actual  power  (flight  test  data)  required  as  recorded 
on  an  instrumented  F-14  during  an  intercept  mission. 
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The  continuously  recorded  flight  data  present  a different  picture 
for  the  frequency  of  major  and  minor  cycles.  Because  the  increase  in 
the  frequency  of  both  cycles  is  significant  and  was  not  originally 
anticipated,  it  is  contributing  to  engine  problems.  The  data  demon- 
strate the  gross  errors  in  synthetic,  sortie  analysis  and  indicate  that 
neither  the  service  nor  the  manufacturer  had  a clear  idea  of  the  prob- 
able pattern  of  engine  operational  usage. 

As  a result,  engine-part  life  has  been  overestimated  and,  hence, 
life-cycle  costs  have  been  underestimated.  By  analyzing  continuously 
recorded  data,  the  original  estimate  of  engine-part  life  was  reduced 
by  an  order  of  magnitude  for  the  first  fan  disk  (see  Fig.  8b).  This 
type  of  information  is  needed  by  engine  designers  and  is  only  avail- 
able through  continuous  recording.  If,  for  example,  continuously 
recorded  data  had  been  available  from  previous  fighter  aircraft  mission 
experience,  the  F-14  powerplant,  an  uprated  model  of  the  TF30,  might 
have  been  designed  differently. 


Before  drawing  any  conclusions  regarding  implications  for  the 
F100  EDS  program,  we  need  to  summarize  the  findings  of  our  case  study 
reviews  and  the  engine  operational  duty-cycle  work.  These  findings, 
shown  in  Fig.  9,  are  as  follows: 

(1)  The  benefits  and  costs  of  engine  monitoring  are  still  very 
uncertain.  Quantitative  benefits  have  not  been  realized,  and  costs 
have  been  higher  than  expected.  Because  the  control  groups  did  not 
control  all  the  variables  of  interest,  and  because  the  time  interval 
for  the  tests  was  too  short,  we  can't  be  definitive  about  the  outcomes. 

(2)  A continuous  recording  system  provides  important  design  in- 
formation that  can  be  of  substantial  value  to  the  Air  Force,  although 
many  of  the  important  benefits  cannot  now  be  treated  quantitatively. 
Time  is  required  to  assimilate  this  information  and  to  develop  and  to 
fully  utilize  the  data  derived  from  the  monitoring  procedure.  Specific 
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action  will  be  necessary  if  we  are  to  obtain  certain  long-term  bene- 
fits, The  maximum  utility  of  the  monitoring  process  occurs  early  in 
an  engine’s  life  when  an  opportunity  still  exists  to  affect  engine 
component  redesign  and  to  give  direction  to  the  component  improvement 
program. 

(3)  Improvements  in  the  EDS  design  should  evolve  as  we  gain  ex- 
perience with  the  system. 

(4)  Monitoring  systems  provide  the  engine  design  and  test  com- 
munity and  maintenance  crew  with  an  understanding  of  problem  causes 
and  effects  and,  through  corrective  actions,  ultimately  improve  the  ma- 
terial condition  of  the  engine.  Engine  overtemperatures  are  especially 
important,  particularly  in  the  case  of  a single-pilot  and  a single- 
engine aircraft. 

(5)  The  increased  sensitivity  of  pilots  and  ground  crews  to 
engine  condition  does  result  in  more  malfunction  reports  and  conse- 
quently increases  costs  initially.  Problems  are  identified  and  re- 
sources must  be  allocated  to  correct  them.  Another  source  ci  increased 
costs  is  the  low  reliability  and  high  false-alarm  rates  experienced 

by  most  of  the  monitoring  systems  during  initial  operations.  Together 
these  two  factors  can  initially  result  in  a low  system  credibility, 
a handicap  difficult  to  overcome. 

(6)  Several  of  the  programs  have  been  dominated  b>  early 
lonitoring-system  hardware  and  software  problems  such  as  latent  design 
deficiencies,  manufacturing  defects,  nonavailability  of  key  software 
subroutines,  and  logic  errors  in  software.  These  problems  continue 

to  exist  even  after  the  monitoring  systems  have  reached  the  field. 

It  takes  a long  time  to  work  out  all  the  lugs,  and  this  drives  up 
initial  support  costs,  especially  when  systems  are  prematurely  fielded. 
These  early  problems  are  difficult  to  overcome,  but,  again,  for  both 
short-term  maintenance  and  longer-term  design  and  testing,  the  bene- 
fits appear  significant  if  they  can  be  resolved. 


IV.  IMPLICATIONS  FOR  F100  F.DS 


DESIGN  OBJECTIVES 

What  does  all  this  mean  for  the  F100  EDS  In  terms  of  design  ob- 
jectives, completeness  of  analysis,  cost  estimates,  and  development? 

Let’3  turn  to  the  last  column  of  Fig.  10,  which  is  the  matrix 
shown  previously  in  Fig.  7.  Based  on  the  F100  EDS  design,  we  would 
expect  good  coverage  in  the  operational  areas.  For  the  first  three 
objectives  under  maintenance,  we  would  expect,  initially,  to  require 
more  maintenance  and  fuel,  not  less.  Over  the  long  term,  the  cost 
outcome  is  uncertain. 

We  would  expect  to  realize  those  benefits  that  are  difficult  to 
quantify,  but,  as  we  have  seen  from  all  the  case  studies,  they  will 
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take  time  to  develop.  In  the  management  area,  for  example,  it  is  not 

clear  how  the  hard-time  limit  on  the  life  of  critical  parts  will  be 
* 

affected.  As  presently  designed,  the  EDS  is  a snapshot  recording 
system  and,  like  previous  U.S.  monitoring  systems,  will  not  capture 
the  long-term  design-oriented  benefits.  Those  benefits  can  be  achieved 
by  developing  a continuous  recording  option. 

The  matrix  illustrates  the  narrow  scope  of  the  original  F100  EDS 
concept  formulation  and  the  follow-on  life-cycle  cost  analysis.  The 
information  originally  provided  to  the  decisionmaker  for  the  F100  EDS 
program  consisted  of  only  the  first  four  objectives  under  maintenance 
in  Fig.  10,  and  the  decisionmaker  was  led  to  believe  that  all  four  of 
them  would  be  achieved.  But  we  have  seen  from  experience  that,  ini- 
tially at  least,  maintenance  costs  will  increase.  The  long-term  out- 
come remains  undefined.  That  the  monitoring  system  has  the  potential 
of  contributing  to  design  improvements,  as  well  as  to  operating  and 
support  cost  reductions,  was  not  emphasized  in  earlier  analyses.  All 
the  objectives  shown  in  Fig.  10  represent  the  type  of  information  the 
decisionmaker  needs. 

ESTIMATED  COSTS  AND  SAVINGS 

Previous  F100  EDS  studies  have  focused  on  the  net  life-cycle  cost 
savings.  In  their  life-cycle  cost  analysis,  the  Air  Force  originally 
addressed  only  the  first  three  cost  elements — RDT&E,  procurement,  op- 
erating and  support — shown  in  Fig.  11.  To  these,  data  management, 
monitoring  system  improvements,  and  F-16  EDS  modification  costs  must 
be  added.  The  first  two  elements  are  generic  to  all  life-cycle  cost 
analyses  and  should  always  be  considered.  This  problem  is  typical  in 
life-cycle  analysis  because,  usually,  only  some  of  the  costs  and  some 
of  the  benefits  are  analyzed.  The  third  element  is  unique  to  the  F100 
EDS  program. 

We  find  that  with  large,  complex  systems,  resources  required  for 
data  management  can  be  extensive.  We  also  find  that  it  will  be 


The  F100  engine  uses  on-condition  maintenance,  where  the  fixed 
time  T30  for  the  entire  engine  will  be  replaced  by  a hard-time  limit 
on  the  life  of  critical  parts  and  individual  modules. 
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ESTiMATED  COSTS  AND  SAVINGS  FOR  FIOO  EDS 
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Figure  11 


necessary  to  make  monitoring  system  improvements  to  increase  the  ini- 
tially low  reliability,  improve  sensor  durability,  and  increase  system 
capability.  Of  particular  concern,  however,  is  the  fact  that  currently 
the  F-16  does  not  have  space  available  for  the  F100  EDS  to  fit  into 
the  aircraft.  The  EDS  may  therefore  have  to  be  repackaged  or  redesigned. 
Even  though  the  F-16  engine  is  considered  to  be  the  same  type/model/ 
series  as  the  F-15  powerplant,  it  may  differ  in  an  important  aspect: 
consideration  is  now  being  given  to  the  use  of  an  electronic  fuel  con- 
trol in  the  F-16  engine,  whereas  the  F-15  powerplant  has  a hydrome- 
chanical control.  This  difference  could  result  in  different  sensors 
and  modifications  to  the  F100  EDS  hardware  and  software  if  the  digital 
electronic  fuel  control  becomes  available. 

IMPLICATIONS  FOR  COST  SAVINGS 

The  Air  Force  anticipated  that  the  acquisition  cost  of  the  F100 
EDS  would  be  more  than  offset  by  outyear  engine  support  cost  savings. 

The  first  three  items  shown  on  the  lower  portion  of  Fig.  11 — fewer 
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unscheduled  engine  removals,  less  fuel  usage,  less  engine  secondary 
damage — reflect  the  sources  of  these  life-cycle  cost  savings.  Pre- 
vious monitoring  system  experience,  as  shown  by  our  case  study  reviews, 
strongly  indicates  that  the  first  two  items  will  not  be  realized,  at 
least  initial),;'.  Some  data  are  just  beginning  to  appear  for  the  third 
item  (engine  secondary  damage),  and  so  we  are  unable  to  address  it. 

To  these  three  expected  sources  of  life-cycle  cost  savings,  a fourth, 
not  previously  considered  directly,  should  be  added.  The  engine 
monitoring  system  should  help  to  extend  the  interval  between  depot 
overhauls.  Even  a small  change  in  this  interval,  affecting  over  3000 
engines,  could  result  in  significant  cost  savings. 

Previous  studies  conducted  by  both  the  Air  Force  and  contractors 
indicated  that  the  F100  EDS  is  expected  to  produce  a net  cost  savings. 
The  difference  between  the  dashed  line  and  the  solid  black  line  in 
Fig.  12  depicts  schematically  the  life-cycle  cost  savings  predicted  by 
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those  studies.  However,  the  results  from  previous  monitoring  system 
experience  indicate  quite  a different  picture  (short  dotted  line) , 
that  the  cost  will  be  greater,  at  least  initially.  There  is  still  a 
great  deal  of  uncertainty  about  outyear  costs  and  whether  or  not  they 
will  indeed  eventually  fall  below  the  solid  or  dashed  lines.  But  if, 
for  example,  we  can  improve  our  understanding  of  failure  cause  and 
effect,  and  thus  extend  the  depot  overhaul  interval  sooner  than  would 
be  possible  without  the  EDS,  reduced  outyear  costs  can  be  achieved 
and  the  system  may  be  cost  effective  over  the  long  term.  There  is  no 
current  evidence  either  way.  The  important  point  is  that  if  cost  sav- 
ings constitute  the  only  criterion  for  EDS,  it  is  doubtful  that  the 
F100  EDS  can  be  justified  on  the  basis  of  current  information. 

PROGRAM  SCHEDULE 

As  shown  by  Fig.  13,  important  decisions  regarding  EDS  develop- 
ment still  lie  ahead.  The  program  is  approximately  5 months  behind 

•J* 

schedule  because  of  hardware  and  software  problems.  A production 
retrofit  decision  is  planned  following  completion  of  the  flight-test 
demonstration  sometime  in  early  1980.  There  has  been  some  discussion, 
however,  that  the  production  retrofit  decision  will  be  made  before  the 
flight-test  program  has  been  completed.  Once  the  production  decision 
is  made,  the  program  will  move  into  a transition  phase,  where  the  sys- 
tem will  be  redesigned  to  take  advantage  of  what  has  been  learned  dur- 
ing the  flight-test  evaluation  phase.  Following  completion  of  the 
transition  phase,  F-15  and  F-16  EDS  production  and  retrofit  are  sched- 
uled to  begin. 

DEVELOPMENT  PLAN 

Figure  14  makes  three  points  about  the  EDS  development  plan: 

(1)  Based  on  current  F100  experience,  we  expect  that  there  will 
be  between  30  and  50  engine  removals  during  the  planned  flight-test 
demonstration.  We  base  this  estimate  on  five  F-15  aircraft  flying  30 


The  shape  of  the  curve  in  Fig.  12  is  taken  from  Ref,  22. 
"^As  of  October  1978. 
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hours  a month  for  10  months  and  on  the  current  engine  unscheduled  re- 
moval rate.  Some  of  the  major  complaints  of  the  earlier  monitoring 
systems  programs  were  that  not  enough  engine  problems  were  experienced 
to  wring  out  the  system.  We  anticipate  that  this  will  be  the  case 
for  the  F-15  as  well. 

(2)  For  the  F100  EDS  to  really  be  effective,  it  must  operate  as 
a system.  Consideration  must  be  given  to  operational  software  as  well 
as  to  hardware,  i.e.,  to  how  the  data  are  to  be  used  and  by  whom.  For 
maintenance  purposes,  the  data  must  be  used  at  the  squadron  and  wing 
level,  whereas  the  engineering  data  can  be  processed  at  a central  site. 
If  data  utilization  for  maintenance  is  not  taken  into  account,  the 
time  required  for  EDS  to  become  fully  operational  will  increase. 

(3)  The  F100  engine  is  a technically  advanced  engine  and  sub- 

* 

stantial  improvements  in  reliability  are  possible.  In  the  process 
of  trying  to  model  this  engine’s  mechanical  and  thermodynamic  behavior, 
many  things  will  be  learned  that  may  cause  a change  in  the  parameters 
monitored  and  thereby  result  in  modifications  to  the  EDS  software  and 
hardware.  Thus,  in  all  likelihood,  several  iterations  of  the  EDS 
design  will  probably  be  necessary  until  the  system  functions  as 
intended. 

PROPOSED  FUNDING  PROFILE 

Figure  15  shows  the  F100  EDS  proposed  funding  profile  for  the 
flight-test  demonstration,  transition  phase,  and  F-15  and  F-16  out- 
fitting. The  RDT&E  phase — previously  the  flight  demonstration  and 
transition  phase — accounts  for  less  than  5 percent  of  the  anticipated 
program  costs  in  then-year  dollars.  The  remaining  costs — production/ 
retrofit — divide  almost  equally  between  the  F-15  and  F-16  aircraft. 

The  funding  profile  reveals  a small  RDT&E  cost  fraction,  the 
leverage  available  if  all  the  aircraft  do  not  need  Instrumentation, 
and  the  short  time  remaining  before  commitment  of  substantial  procure- 
ment funds.  Since  the  hardware  procurement  and  installation  costs 

* 

Current  F100  engine  removal  rate  is  greater  than  10  per  1000 
engine  flying  hours,  as  compared  with  3 per  1000  for  the  J79. 


wurnmm 


-28- 


HOO  EDS  PROPOSE)  FUNDING  PROHLE 

(OCTQQSl 1978)** 


111 


BFUGHT  DEMONSTRATION/ 
TRANSITION 

PRODUCTION/RETROFIT 
H F-lf 
EJF-16 


i 


♦Th»n-y*ar  dollars 
•Being  revised 


V 

Hi 


77  78  79  80  81  82  83  84  85  86  87  88 

FISCAL  YEAR 


Figure  15 


constitute  such  a large  percentage  of  the  program  costs,  obtaining  and 
verifying  the  optimum  EDS  configuration  and  the  number  of  aircraft  to 
be  instrumented  should  be  of  paramount  concern. 

CONCLUSIONS 

There  has  never  been  a controlled  experiment  of  a maturing  engine 
over  a long  enough  time  period  to  allow  monitoring  system  outcomes  to 
be  quantified.  However,  several  conclusions  can  be  drawn  based  on  our 
case  studies.  These  are  outlined  in  Fig,  16. 

We  conclude  that  the  maintenance  cost  savings  used  to  justify  the 
F100  EDS  are  unlikely  to  materialize  over  the  short  term.  But  whether 
EDS  passes  or  fails  in  the  narrow  sense  of  ooet  savings  over  the  short 
tern  should  not  he  the  sole  criterion  on  which  EDS  is  judged.  Sub- 
stantial value  lieB  in  the  potential  benefits  of  (a)  anticipating 
needed  maintenance,  (b)  helping  maintenance  crews  and  engineering 
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Figure  16 


support  personnel  to  better  understand  cause  and  effect,  of  engine 
failure,  and  (c)  verifying  that' maintenance  has  been  properly  performed. 
These  benefits  are  especially  significant  now  that  the  Air  Force  is 
moving  to  an  on-condition  maintenance  posture,  as  is  envisioned  for 
the  F100  engine.  They  can  also  be  important  in  helping  to  achieve  the 
original  design  objectives  of  the  EDS.  Unfortunately,  none  of  these 
benefits  can  be  quantified  on  the  basis  of  experience  to  date. 

As  currently  designed,  the  development  plan  is  unlikely  to  yield 
conclusive  evidence  on  the  value  of  EDS,  because  the  test  aircraft  sam- 
ple is  too  small  and  programmed  flight  hours  are  too  few.  In  several 
important  areas,  the  program  focus  is  too  narrow.  For  example,  the 
software  and  data  processing  are  not  considered  within  the  overall  sys- 
tem, and  the  long-term  design-oriented  benefits  are  not  given  proper 
consideration. 
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In  its  present  limited  scope,  the  EDS  program  omits  valuable  long- 
term returns,  such  as,  for  example,  long-term  design  feedback  and  im- 
provement in  test  cycles*  This  type  of  information  should  help  the 
Air  Force  in  its  component  improvement  program  for  the  F100  engine  in 
both  the  F-15  and  F-16  applications,  as  well  as  in  future  engine  design 
programs.  It  is  especially  important  now  that  reliability,  durability, 
and  cost  issues  are  almost  on  an  equal  footing  with  performance. 
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V.  RECOMMENDATIONS 


We  recommend  that  the  Air  Force  continue  to  develop  a turbine  en- 
gine monitoring  system  for  the  F100  engine  but  that  certain  actions 
be  taken  before  a commitment  of  a quarter  of  a billion  dollars  is  made 
to  outfit  all  F-15  and  F-16  aircraft  with  the  EDS,  These  actions  are 
outlined  in  Fig.  17. 
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Figure  17 


First,  because  a great  deal  of  uncertainty  exists  about  the  cost 
and  benefit  outcomes,  we  recommend  a flexible  phased- implementation 
schedule  that  will  expand  the  test  program  and  defer  a full-scale  pro- 
duction go-ahead.  Not  only  will  phased-implementation  maintain  program 
flexibility,  but  it  will  also  provide  time  to  establish  a control  group 
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to  control  all  the  variables  needed  for  a more  complete  understanding 
of  the  costs  and  benefits  of  monitoring  the  FIDO  engine*  Hardware 
and  software  problems  could  then  be  experienced  in  a small  sample  of 
aircraft  rather  than  accrue  over  the  entire  F-15  i.nd  F-16  aircraft 
inventory . 

The  additional  time  gained  with  a phased-implementation  schedule 
could  be  used  to  reconsider  the  objectives  of  the  F100  EDS  program, 
since  several  of  the  important  benefits  identified  in  the  matrix  can 
be  satisfied  by  instrumenting  a small  sample  of  the  F-15  and  F-16  in- 
ventory- This  option  should  be  included  as  a possibility  in  the  de- 
cisionmaking process. 

Because  important  differences  exist  between  the  F-15  and  F-16  in 

both  engine  installation  and  mission  flown,  it  is  recommended  that  the 

* 

F-16  application  be  fully  integrated  into  the  F100  EDS  program. 

Second,  the  scope  of  the  EDS  should  be  broadened  to  include  the 
valuable  contribution  that  information  feedback  can  make  to  the  designer 
over  the  long  term.  Of  particular  importance  is  the  correlation  be- 
tween testing  and  operational  duty  cycles  for  the  F-15  and  F-16,  since 
their  mission  profiles  will  be  quite  different  and  the  engines  for 
each  application  should  be  tested  to  the  relevant  duty  cycles.  For 
both  F-15  and  F-16  applications,  a continuous  recording  option  and  a 
small  staff  dedicated  to  analyzing  the  data  obtained  are  very  important. 
As  currently  designed,  the  EDS  senses,  but  does  not  save,  the  data 
necessary  for  improving  future  engine  design,  A small  device  to  save 
the  data  would  be  sufficient  to  provide  the  necessary  design  informa- 
tion. This  device  needs  to  be  installed  only  on  a handful  of  aircraft 
at  each  operational  location. 

Third,  the.  life-cycle  analysis  should  be  revised  to  reflect  more 
completely  the  costs  of  and  benefits  provided  by  the  EDS  relative  to 
the  aircraft  as  well  as  to  the  engine.  The  costs  and  benefits  of  vari- 
ous EDS  options  could  then  be  estimated  more  realistically. 

*If  aircraft  use  is  changed,  the  operational  duty  cycle  for  the 
engine  must  be  redefined,  because  a change  in  usage  will  cause  differ- 
rent  failure  modes  in  the  engine. 
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